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ABSTRACT 

This study attempted to develop a reliable ?.nd valid 
instrument for assessing work environment and continuous quality 
improvement efforts in the non-academic sectors of colleges and 
universities particularly those institutions who have adopted Total 
Quality Management programs. A model of a work environment for 
continuous quality improvement was developed and an instrument was 
designed to assess non-academic staff's perceptions of the quality 
process, practices, and results at the University of Michigan (UM) . A 
literature review ident i f i ed* ma j or organizational and higher 
education models of quality improvement. Researchers designed a study 
instrument of 217 items which was evaluated by pilot testing with 
focus groups of UM staff. The final instrument consisted of 201 items 
that incorporated 35 "quality culture and climate" dimensions. Items 
focused on perceptions at the respondents' immediate work "unit" 
level. All permanent non-academic staff completed *he questionnaire 
in February 1994. Of 10,400 questionnaires distributed 4,900 were 
completed and returned. Results were subjected to data reduction, 
index development, and reliability and validity testing from which 27 
factors emerged. Reliability for indices ranged from .53 to .96. 
Extensive appendixes contain details of methodology, factor analyses, 
indices descriptions, and comparisons of conceptual categories and 
factor indices of a quality culture and a quality climate. (Contains 
44 references.) (JB) 
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abstract 

Despite the enormous amount of energy that colleges and universities spend on 
implementing quality improvement initiatives, few institutions have developed 
methods for assessing the work environment for continuous quality improvement. 
The goals of this study are to provide administrators and institutional researchers 
with a means for better understanding the climate and culture of their institution's 
non-academic work environment and for assessing continuous quality 
improvement initiatives. This paper reports on a research study which developed 
a reliable and valid instrument for assessing the climate and culture of a 
university's work environment for continuous quality improvement. 

A model of a work environment for continuous quality improvement was 
developed and an instrument was designed io assess non-academic staffs 
perceptions of the quality processes, practices, and results within this 
environment. The survey results were subjected to data reduction, index 
development, and reliability and validity testing. The implications of the findings 
for institutional use and further research are discussed. 
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Assessing the Culture and Climate 
for Quality Improvement in the Work Environment 



i. Introduction 

Colleges and universities from all sectors of education have 
implemented a variety of institutional initiatives for continuous quality 
improvement (Seymour, 1993). The purpose of these initiatives has been to 
improve the overall quality of the institution. Yet few postsecondary institutions 
have attempted to systematically assess the work environment designed 
around the practices, processes, and values implicit in a climate or culture for 
continuous quality improvement. A review of the literature on quality in higher 
education shows little consensus regarding the concepts and dimensions which 
constitute continuous quality improvement in postsecondary education. The 
literature also includes no systematic quantitative research conducted on 
quality improvement dimensions, and therefore, no empirical foundation exists 
for examining the initiatives. 

The purpose of this paper is to develop a research based instrument for 
assessing the climate and culture for continuous quality improvement among 
non-academic staff in higher education. The paper is based on a research 
project which examined the work environment perceptions of non-academic 
employees at the University of Michigan. A framework for examining the 
institutional activities, processes, and value dimensions in a work environment 
stressing continuous quality improvement is developed. The study describes 
the development, the design, and pilot testing of a survey instrument, the 
process of data reduction and index development and the broader implications 
for higher education institutions. 
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||. RESEARCH QUESTIONS 



The primary research questions addressed by this study are: 

1 What is an appropriate conceptual framework for examining the work 
environment for continuous quality improvement in a higher 
educational setting? 

2 What are the appropriate dimensions which constitute a culture and 
' climate for continuous quality improvement in that environment? 

3 Can reliable and valid measures and indices reflecting the 
dimensions be developed in an instrument designed to measure 
participants' perceptions of their work environment? 



III. LITERATURE REVIEW 



The literature relating to total quality management (TQM) in business and 
industry TQM or continuous quality improvement as applied to the higher 
education setting, and organizational culture and climate was reviewed for th.s 
study These publications were used to identify a the elements or d.mens.ons of 
a cuKure or climate for continuous quality improvement within the higher 
education work environment. The final purpose of this literature review was to 
find current examples of quality improvement dimensions outlined for higher 
education and determine the extent of systematic quantitative research that has 
been conducted in defining these dimensions. 

Overview of Total Quality Management 

Total Quality Management is a system's oriented approach to managing 
the continuous improvement of institutional quality. It is grounded in dual lines 
of research- statistical quality control and behavioral science studies of 
organizations (Kahn, 1993). In the early 1930*8. Walter Shewhart (Keller, 1992) 
began to use sampling and statistical techniques to "control quality into" the 
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products at Bell Laboratories. The importance of systematic ptocess 
improvement was realized and the era of statistical quality control was born 
(Garvin, 1988). Doming (1986), Juran(1988), Crosby (1988), Feigenbaum 
(1991), Imai (1986), and Ishikawa (1982) advanced and broadened this 
statistical approach to quality by integrating human relations and organizational 
aspects of quality management. The period of the 1950's to the 1980's 
became known as the quality assurance era as industry in the United States 
began to adopt and implement quality improvement initiatives (Seymour, 1993; 
Garvin, 1988). 

Higher Education and Continuous Quality Improvement 

As large corporations in the United States like Ford Motor Company, 
Xerox, and Procter and Gamble began to achieve success by utilizing quality 
improvement principles, institutions of higher education began to look at TQM 
as a means of remaining viable in an increasingly competitive environment 
(Robinson, Akers, Artzt, Polling, Galvin, & Allaire, 1991 ; Peterson, 1992). 
Concurrently, several external reports critical of higher education led calls for 
increased accountability for college end universities. These reports, which 
included A Nation at Risk: The Imperative for Educational Reform (U.S. 
Department of Education, 1983), Involvement in Learning: Realizing the 
Potential of American Higher Education (National Institute of Education, 1984), 
To Reclaim a Legacy (Bennett, 1984), and Integrity in the College Curriculum 
(American Association of Colleges, 1985), challenged academic leaders in 
higher education to more closely assess the results of educational processes 
(Peterson, 1993). 

In 1989, leaders in professional schools and community colleges began 
to write about the application of Total Quality Management to higher education 
and university leaders soon followed (Peterson and Cameron, et. al., 1993). 
Quality became viewed as an opportunity and emphasized as a strategic 
management function. It was included in strategic planning processes and 
began to become "managed in" (Garvin, 1988). As this began to happen, the 
conception of quality shifted from being defined by the organization to being 
defined by the customer. This led to multi-dimensional and sometimes mutually 
competing views of what constituted quality, given the multiple stakeholders in 
higher education (students, faculty, staff, alumni, and the public). The 
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importance of focusing on the needs of stakeholders, institution-wide 
involvement, and group problem solving and decision-making was common in 
higher education and already a well developed in the behavioral science 
research on organizations (Kahn, 1993). Higher education has begun to 
embrace the principles of quality management and examples of quality 
improvement initiatives can be found across all sectors of institutions (Seymour, 
1993). While college and university quality initiatives are increasingly 
numerous, they are primarily limited to the administrative and business 
functions of the institution, and assessments of the nature and results of these 
initiatives have been limited. 

Work Environment and Organizational Culture and Climate 

An organization's work environment can be portrayed in several ways: 
1) as objective patterns of behavior or working conditions; 2) as the perceived 
patterns of behavior and attitudes related to that environment; or 3) as the 
underlying values and beliefs of the organization or its participants. These 
concepts are often referred to as the organization's culture or climate and 
provide members with an understanding of the meaning of their organization 
and their internal work environment (Peterson, Cameron, Jones, Mets, and 
Ettington, 1986; Peterson and Spencer, 1990). Culture and climate also 
provide a framework for members of the organization to make sense of the 
nonrational and informal aspects of their institutional environment .Peterson 
and Spencer, 1990). 

Schein (1985) defines culture as the basic assumptions and beliefs that 
are shared by the organization. Dennison (1990) states that organizational 
culture is comprised of shared values, beliefs, and principles. Specifically 
focusing on higher education, Peterson and Spencer (1990) term institutional 
culture as the "organizational glue" that holds the organization togevher. 
Chaffee and Tierney (1988, p.5) define institutional culture as "the collective 
values held by members of the organization [which] derive [their] force from the 
traditions, processes, and goals held by those most intimately involved in the 
organization's working." In higher education, cultural values are inherent in the 
institution's history, tradition, academic mission, governance processes, 
administrative methods, and delivery processes (Keller, 1983; Chaffee and 
Tierney, 1988; Peterson and Spencer, 1990; and Austin, 1990). 
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Climate is defined (Peterson, Cameron, Jones, Mets, and Ettington, 
1 986) as the "current, common patterns of important dimensions of 
organizational life or its members' perceptions of and attitudes toward them." 
Institutional climate is also defined (Schneider and Rentsch, 1988) as 
organizational policies, practices, and procedures that communicate the goals 
that are important to an organization and create a sense of institutional 
imperative. Peterson and Spencer (1990) contrast climate as the 
organizational "atmosphere" with culture as the organizational "values". 

Organizational culture and climate affect attempts to implement quality 
improvement initiatives in higher education. Incorporating these initiatives 
requires integrating the elements of improvement initiatives into both the culture 
and the climate—the beliefs and the practices— that frame all institutional efforts 
(Bemoski, 1991; Coate, 1991; Seymour and Collett, 1991). Seymour (1993) 
has identified a "culture of quality" in which members develop, share, and 
continually reinforce a common understanding of what qualify is and how to 
pursue it. Cameron (1994, p. 15) defines an organization's quality culture as 
"the general orientation or definition of quality adopted by an organization." 
This general orientation consists of the way members of an organization think 
about quality, approach quality, and define its main objective. In his study of 68 
automotive companies, Cameron found these organizations "differ in terms of 
their values, assumptions, beliefs, and meaning (i.e., their culture) regarding 
quality." Since culture provides the organizational context in which all activities 
are performed, quality culture is manifest in the organization's policies, 
practices, and procedures which comprise organizational climate (Cameron, 
1994). Thus the literature provides a conceptual view of the culture and climate 
on which to build an empirical foundation for examining continuous quality 
improvement in the industry. However, there is little consensus on the 
dimensions which relate to a culture or climate for quality improvement in higher 
education. 

Quality Dimensions in Business and Industry 

Table 1 shows the many sources that have been instrumental in 
describing the elements and suggesting the dimensions of organizational 
quality management. Deming (Walton, 1986) outlines fourteen points that he 
identifies as the basic principles of quality management. These fourteen points 
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form a "philosophy of management", with which statistical control methods are 
consistent (Walton, 1986). They have widely contributed to the formation of 
dimensions of quality indicators across industry sectors (Miller, 1991). 

A second major source is the work of the Juran Institute for Quality. It's 
primary focus is on the dimensions of planning functions and customer 
satisfaction for both product and service oriented organizations (Juran, 1 992). 
The quality dimensions outlined by this Institute are framed in the Juran Trilogy 
of quality planning, quality control, and quality improvement (Juran, 1992). 

A third source of dimensions is the Malcolm Baldridge National Quality 
Award Program (National Institute of Standards and Technology, 1994). This 
award is made annually to manufacturing companies, service companies, and 
small businesses which are recognized as excelling in quality management 
and quality achievement. 

A fourth source which has been widely utilized in industry is entitled 
ISO9000. ISO9000 certification demonstrates the ability of an organization to 
control processes that determine the acceptability of the service provided 
(Rabbit & Berg, 1993). The ISO9000 criteria are focused on the data collection 
and product control. They include use of statistical methods,, quality records 
and documentation, after-sales servicing, control of processes, and quality 
costs. 

A fifth source is the Crosby Perspective (Crosby, 1 980) which illustrates 
the key actions necessary in implementing a quality management program. 
This perspective goes beyond production management and into office 
management and includes dimensions such as management commitment, 
teamwork, measurement, communication, training, and goai setting. 

The Cameron model is a sixth source. Cameron (1994) utilized the 
Baldridge Award criteria to supplement his self-developed dimensions of quality 
improvement to conduct two different studies assessing quality values, 
processes, and practices in the work environment in approximately one 
hundred businesses associated with the automotive industry. These studies 
offered an empirical foundation for examining dimensions of an organization's 
work environment which stresses a culture and climate of quality. These 
seminal works, along with numerous other more minor sources, were reviewed 
to identify the various perspectives and elements of total quality management 
which have been advanced. 

[INSERT TABLE 1] 
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Quality Dimensions in Higher Education 

Peterson (1982, p.188) states that, "Higher education institutions are 
justly criticized for poor management or inefficiency; they seize a tool, 
technique, or approach from another setting (often business or government) 
without examining how it worked there or whether can work without 
modification; and they become disenchanted and reject it or go through 
agonizing reappraisal." Table 2 shows several higher education authors who 
have attempted to identify dimensions of continuous quality improvement as 
applied to higher education. Marchese (1991) identified twelve "TQM themes" 
present in improvement initiatives in colleges and universities. Sherr and 
Lozier (1991) outline five dimensions of a "complete theoretical system" of 
quality improvement in higher education. In a survey of twenty-two institutions 
he identified as "pioneering colleges and universities" which had implemented 
quality improvement initiatives, Seymour (1993) identified ten dimensions of 
quality improvement. Chaffee offered eight dimensions in her work written in 
1990. Qualt.ec and the University of Michigan developed eight dimensions 
characterizing the key elements necessary for a continuously improving 
university. Sylwester and Harris (1991) narrow the dimensions of quality 
improvement initiatives to three keys to successful cultural adjustment. 

The review of the literature related to quality improvement in higher 
education reflected that there had been no systematic research conducted on 
quality improvement dimensions. The numerous definitional elements listed in 
both the business and the higher educational models provided a foundation for 
a dimensional analysis of continuous quality improvement in higher education. 
From a content analysis of the works cited above and over one hundred other 
articles related to total quality management in business and industry and 
continuous quality improvement in higher education— annotated in Total Quality 
Management in Higher Education: From Assessment to Improvement 
(Peterson, Cameron, et. al., 1993)— thirty-five elements of TQM in higher 
education have been identified. These elements of quality reflect the values, 
principles, practices, techniques and management strategies and other 
fundamental ideas which have been suggested by these works. Appendix 1 
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lists the thirty-five elements of continuous quality improvement identified in this 
review and analysis. 

[Insert Table 2] 



IV. Conceptual Framework 



Based on the literature review and the preceding content analysis, broad 
dimensions for examining a quality oriented higher educational work 
environment were identified. The thirty-five elements of continuous quality 
improvement were clustered into thirteen conceptual categories of a culture and 
climate for quality improvement in the work environment developed by 
Cameron (1994). 

A conceptual framework was then developed for examining the 
contextual influences which the literature suggests might affect the culture and 
climate of the work environment and the results which might be expected from a 
quality oriented work environment (see Figure I). The conceptual framework 
which is divided into four main areas: 1) external influences on the unit work 
environment, 2) culture: quality oriented values and philosophy, 3) climate: 
quality oriented work processes and support practices, and, 4) expected 
outcomes or impact of a quality oriented unit work environment. (These are 
discussed \r, the following sections). The central conceptual component of this 
model focuses on employee perceptions of their immediate work environment's 
quality culture and its climate of quality oriented processes and practices. 
Those include the thirteen dimensions which are indicators of the culture and 
climate for quality in higher education institution's non academic work 
environment. The development of an instrument to measure the quality 
oriented culture and climate of a college or university's non-academic 
employee's work environment is the central focus of this paper. 



Influences on the Unit Work Environment 



The climate and culture of the unit work environment may be influenced 
by a number of external factors. The conceptual model developed for this 
project suggests that there are four major sources which combine to affect 
employee's perception of the unit work environment. The influences on the unit 
work environment are indicated by four domains listed under the label of "other 
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environments" in Figure 1. First various pressures and dynamics of the 
environment external to the institution may influence the unit work environment 
and includes things such as the broader social, political, economic and 
competitive context and the external institutional environment. A second major 
source which affects the climate and culture of the unit work environment 
includes the characteristics of the institution in which the unit is a part. 
Institutional characteristics such as the mission, values, size, type, history, and 
organization of a college or university will influence the work environment of the 
unit. The third major source which may influence relates to the characteristics of 
the work unit itself. These characteristics include the function of the unit, the 
organizational design of the unit, the unit's relationship to other units both 
horizontally and vertically within the institution, the size of the unit, and the 
leadership of the unit. Finally, there are individual factors such as employee's 
gender, ethnicity, race, education, training, years at institution, etc. which may 
influence their perceptions of the unit work environment. 

Cultural Categories: Quality Oriented Philosophy and Culture 

The quality improvement dimensions which emerged from the literature 
were characterized as being either a part of a culture reflecting quality oriented 
values and philosophy or as a climate reflecting the emphasis on quality 
oriented work processes and support practices. The culture and climate of the 
work environment are the center of this model. Culture dimensions were 
organized into three conceptual categories identified as being integral to a 
unit's quality oriented culture and philosophy: 1) quality philosophy, 2) quality 
improvement culture, and, 3) organizational culture (Figure 1). 

Quality Philosophy Quality philosophy refers to the way staff members 
in a particular unit fundamentally feel about continuous improvement and the 
wa*' they think about quality in their work environment. Quality philosophy 
includes staff members perceptions of the role of quality in their unit's mission 
and purpose, willingness to change and improve, propensity to interact and 
share success stories, priorities relating to quality, and concerns for qua'ity 
sendee. It is grounded in a philosophy of continuous quality improvement or a 
willingness to change. 
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Quality Improvement Culture Quality improvement culture focuses on 
the embedded understandings and the general values, beliefs, or customs 7 
relating to a concern for continuous improvement. It includes service 
orientation, leadership for improvement, teamwork and collaboration, staff 
development, downsizing approaches, and other key measures of continuous 
quality improvement. 

Organizational Culture Organizational Culture focuses on an overall 
values and underlying standards which reflect the way employee's perceive the 
organizational nature of their unit. It includes dimensions related to the unit's 
dominate organizational characteristic, leadership style, criteria for success and 
management style (Cameron 19 ). 

Climate Categories: Quality Oriented Work Processes & Support 
Practices 

In addition to cultural categories, ten conceptual categories describing a 
unit's climate were identified (Figure 1). Climate categories relate to a unit's 
quality oriented work processes and support practices which are integral to 
continuous improvement. They include: unit climate, planning for improvement, 
satisfying those served, work processes, collecting and using information, unit 
efficiency, leadership, unit staff members, improving performance, and unit 
performance indicators. 

Unit Climate Unit climate refers to the general atmosphere or mood 
within the work unit. It focuses on how employees feel about their daily 
interactions with co-workers, superiors and the work itself. Unit climate refers to 
general dynamics in the workplace including a sense of community, 
cooperation, teamwork, trust, and enjoyment. Unit climate also refers to 
freedom from fear and criticism, positive interpersonal and inter-unit 
relationships, and a feeling of being valued, accepted, and sought after for 
input. 

Planning for Improvement Planning, improvement, and innovation are 
the focus of this category which refers to activities and understandings of unit 
reflecting those concerns. Specifically, it refers to the extent to which a unit has 
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improvement plans in place that are known, understood, and utilized by 
individual unit members. It addresses the involvement that individuals have in 
planning and the level to which planning has been personalized. The unit's 
ability to continuously improve its planning is also included. A unit's focus on 
innovation includes the degree to which employees seek innovation, are 
rewarded for successful developments, and are not punished for taking risks or 
making mistakes. 

Satisfying those Served Service to the unit's clients, service recipients, 
customers, beneficiaries, stakeholders, or constituents, is the focus of this 
dimension. Satisfying those served by the unit encompasses the identification 
of internal and external groups, the understanding of their needs and 
expectations, the use of procedures to gather their input, the utilization of the 
information for improvement, and a general concern for anticipating and 
satisfying those who are served. 

Work Processes The work processes dimension examines the key 
elements of improving the design of basic day-to-day work processes. It 
includes a concern for understanding and improving aii the unit's processes 
which are problematic or problem-free. The category focuses on process 
assessment, efficiency, and effectiveness. This includes using systematic 
methods, front-line staff members, and problem solving teams for assessing 
why mistakes occur and how problem-free processes can be improved. The 
dimension also examines the scope and effectiveness of process improvement 
and the degree to which a holistic approach is taken based on information from 
those being served. 

Collecting and Using Information The collecting and using information 
category examines the scope, collection efforts, and effective use of data and 
information. It assesses the degree to which information is collected and 
utilized in areas such as: individual and unit success, achievement, and 
innovative ideas; timeliness, mistakes, and complaints; employee performance; 
customer and employee satisfaction; benchmarking; and the data collection and 
utilization process itself. The dimension also examines the clarity and utilization 
of quality indicators or standards and the degree to which those moot 
responsible utilize the information. 

AIR Paper . . 
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Unit Efficiency This category relates to the ways in which work is 
organized to increase the unit's efficiency. Unit efficiency focuses on the 
reduction of time and resources used to accomplish a unit's work. It includes 
reducing unproductive employee time; reducing the cycle time to respond to 
those served; reducing costs and waste; reviewing, reorganizing, and 
simplifying work processes; and developing time standards for work activities. 

Leadership The leadership category examines unit leaders' personal 
approach to and involvement in developing a unit which continuously improves 
quality and has a clear understanding of quality values. This includes the 
degree to which the leader communicates and stresses the importance of 
quality improvement, manages by facts, is open to new ideas and change, 
removes barriers and obstacles, reallocates resources based on quality values, 
and leads by example. 

Unit Staff Members The unit staff members category examines how the 
unit's human resources are enabled to develop a continuous quality 
improvement philosophy and to reach their full potential as contributors. For 
example, it focuses on staff member opportunities to participate in education or 
training on total quality and work improvement concepts and techniques, 
rewards for staff members for achieving quality objectives, staff member 
opportunities to participate in personal and professional growth and 
development activities, and improvement of human resources development 
programs. This category also examines the extent to which conditions in the 
unit are conducive to full participation, to staff members being treated with 
dignity and respect and to sensitivity to one another's needs. Further, it focuses 
on staff members having the necessary knowledge and resources to 
accomplish their work. 



improving Performance This category focuses on morale, group 
dynamics, and the working relationships among staff members as they attempt 
to improve performance. Issues such as trust, loyalty, helpfulness, criticism, 
communication, and openness between staff members are examined. It also 
examines emphasis on time standards, reducing waste, improving and 
measuring of performance improvement. 
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Unit Performance Indicators The unit performance category refers to 
how a unit's performance compares to other similar units. It looks at 
perceptions of and information about whether the unit is performing at a level 
higher or lower than a peer unit. Four areas of comparison are of interest: 
overall performance, rate of improvement, number of errors and mistakes, and 
costs of services. 

impact of a Quality Oriented Work Environment 

The outcomes or impact of a culture and climate for quality improvement 
in a unit's work environment are suggested in the literature and are shown in 
the conceptual framework (Figure 1). The model identifies characteristics of the 
units products and services, such as quantity, quality, speed, timeliness, and 
cost; overall unit performance indicators such as employee satisfaction, 
customer satisfaction, comparative results, structure and design and innovation; 
and broader institutional benefits or outcomes. 

Thus, a comprehensive review of the literature yielded thirty-five quality 
dimensions which were then clustered into thirteen broad conceptual 
categories of a culture and climate for quality in the work environment. The 
thirteen culture and climate categories constitute the unit work environment 
within a conceptual framework suggesting contextual influences on and 
outcomes of that work environment. 

v. Methodology 

Study background 

In 1989, the University of Michigan began the development of an institution- 
wide continuous quality improvement initiative designed to "pursue a problem- 
solving approach that would enlist the energy, intelligence, and imagination of 
administrators and staff in improving.. .work systems (University of Michigan, 
1 994)" entitled "M-Quality." M-Quality: Continuous Improvement at The 
University of Michigan (1992) focused primarily on the administrative functions 
of the University, and outlined a long-term strategic quality improvement 
initiative to be implemented in three stages over four years. This plan was 



AIR Paper 



13 



adopted by the University of Michigan in 1992. The first stage was designed to 
integrate the initiative into the "existing organizational structure of the university 
(U of M, 1993)." The second stage is projected to implement continuous 
improvement activity more broadly across the campus. The final stage will 
commence in 1996, be an ongoing process and will seek to solidify the gains 
of the first two stages and to integrate quality improvement activities into the 
daily work of non-academic employers across the University community. 

The University is currently in the first stage of implementation and has 
identified four primary principles as the foundations of the M-Quality initiative. 
Those are (UM, 1994): 1.) pursuing continuous improvement, 2.) managing by 
fact, 3.) respecting people and ideas, and 4.) satisf ying those we serve. The 
design of an assessment component and an evaluation strategy is part of this 
initial stage. 

During the Fall of 1993, the U of M's Quality Leadership Team 
commissioned the development of an evaluation framework and to design an 
instrument to assess nonacademic staff members perceptions of their work 
environment. This instrument will provide an ongoing mechanism for assessing 
the impact of the University's M-Quality improvement-related initiatives. The 
Work Environment Research Group at the Center for the Study of Higher and 
Postsecondary Education (CSHPE) was asked to design an evaluation 
framework, to develop an analytic instrument for measuring the non-academic 
staff member's perception of their work environments emphasis on continuous 
quality improvement values, processes, and practices, and to conduct an initial 
survey of the approximately 10,400 non-academic staff on the Ann Arbor 
campus; and to prepare a summary report of the results. The instrument 
development and testing in this survey are the focus of this paper. 

Questionnaire Development/ Administration 

The survey instrument developed and tested in this study was designed 
in three stages. First, a literature review (reported earlier) identified major 
organizational and higher education models of total quality or continuous 
improvement. These models and other literature were content analyzed and 
the thirty five dimensions of total quality improvement were identified. These 
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dimensions were arranged across the thirteen conceptual categories of a 
culture and climate for quality identified in the conceptual framework. 

In the second stage, an instrument was designed which contained 217 
items reflecting all the quality dimensions and categories (see Appendix 1 ). 
Focus groups representing a cross-section of the University's non-academic 
staff provided feedback on both the construction and content of the instrument. 
Pilot testing with a random sample of employees representing all job families 
and functional areas within the institution was used to further refine the 
instrument. Half of those were interviewed while taking the survey to determine 
their reactions to the instrument directions, length, clarity, section format, 
individual items, item redundancy, and clarity instrument response scales. The 
other half of the pilot respondents took the survey without interruption and were 
then interviewed for their feedback regarding the aforementioned evaluation 
criteria. This feedback was integrated in the final version of the instrument. 

The instrument consists of 201 items that incorporate the 35 "quality 
culture and climate" dimensions identified in the literature. The items are 
distributed over 13 conceptual categories as sections of the questionnaire 
previously described in the conceptual framework (Figure 1). Eleven of these 
categories utilized a Likert response scale with six response choices ranging 
from "strongly disagree" to "strongly agree", "never to always", or "much lower 
to "much higher". The scales in two of the quality culture categories (unit 
improvement orientation culture and unit culture) required dividing 100 points 
among four offered scenarios to accurately reflect the respondent's perception 
of the unit's work environment. 

Items in the questionnaire focused on perceptions at the respondents 
immediate work "unit" level. This was done for several reasons: First, 
respondents are more knowledgeable at this level and their responses are 
likely to be more accurate. Second, focusing at the unit level allowed for 
differentiations among the various work units and functional areas of the 
institution. Finally, the diverse and decentralized nature of the institution was 
more accurately portrayed at this level but could still be aggregated. 

The questionnaire was administered to all permanent non-academic staff 
at the main campus with half-time or greater appointment during February, 
1 994. A follow-up post-card was mailed to all respondents two weeks after the 
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initial ^instrument was sent. 10,400 questionnaires were distributed and 4,900 
usable questionnaires (47.4%) were completed and returned. Table 3 shows 
response rates by job family and functional care of the university. Data 
gathered from this survey provides the basis of this analysis and is the focus of 
the remainder of this paper. 

[Insert Table 3} 

Analysis 

Data were entered with one hundred percent verification. Item level 
descriptive statistics were computed. The means, standard deviation and 
frequency distribution were examined. Cross tabulations compared the 
response patterns by job family, functional area and certain individual 
characteristics. These results are not discussed in this paper). 

To test the content validity of the items arrayed by conceptual categories 
of culture and climate (see Figure 1}, two groups of factor analyses using a 
varimax rotation were conducted. In the first analysis all questionnaire items 
with similar response scales were included. This factor analysis failed to 
converge. Second, factor analyses were run on each of the thirteen conceptual 
categories separately. 

Based upon these results, twenty-? even factors emerged. All 
questionnaire items were included in the factor on which it had the highest 
loading (above .40). The items in each factor were examined for content 
similarity. The results section discusses the indices. Coefficient alpha reliability 
testis were conducted on each of the twenty-seven indices. Reliability's for 
these indices range from .53 to .96. The results of the factor analysis and the 
reliability of the factors are shown in Appendix 2. Appendices 3 and 4 include 
definition of the factors and comparison of the conceptual categories and factor 
indices. 

VI. Results 

Indices of A Quality Culture 

Ten culture indices emerged from the factor analyses of the three 
conceptual categories of culture. Their alpha coefficients were somewhat 
lower than those of the seventeen climate indices, with a range of .53 to .81 . 
AIR Paper 26 
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striving for improvement in the areas of leadership, customer service, cost 
reduction, and teamwork. 

Five indices emerged from the factor analysis of the conceptual category 
related to support for the Unit Organizational Culture category. The first of these 
indices is Clan Culture orientation. The Clan index (reliability .53) relates 
directly to staff members perception that their unit has a strong sense of 
cohesion and teamwork. This index is characterized by team-oriented 
management style, loyalty and commitment, teamwork, and a mentor oriented 
leadership style. 

The second index from the Organizational Culture category is Adhocracy 
(reliability .58). This index relates to a loosely coupled unit culture and focuses 
on innovation and creativity. Its elements include an innovative leadership 
style, unit success defined by innovative efforts, and a management style 
characterized by individual initiative. 

Hierarchy (.58) is the third index in this Organizational Culture category. 
It relates to a bureaucratic unit culture found in more traditional organizational 
structures. This index focuses on a security oriented unit management style, a 
focus on policies and procedure that promote stability, success defined by 
efficiency and stability, and an organizing leadership style. 

The fourth index from the Organizational Culture category is the Market 
culture indsx (reliability .71). This index relates to a results oriented unit culture. 
It focuses on an achievement oriented management style, a hard-driving 
leadership style, success defined by competitiveness, and goal 
accomplishment as the unifying cultural factors. 

The final index from the Organizational Culture category is a single item 
on Change. This relates to a unit's emphasis on change. It is characterized by 
a change orientation ranging from leading, adapting, responding, to resisting 
change within the unit. 

Indices of a Quality Climate 

Seventeen climate indices emerged '40m the ten factor analysis of each 
of the ten conceptual categories of climate. The coefficient alpha for the indices 
ranged from .83 to .96. This level o* reliability indicates that the climate indices 
reflect strong dimensions or indices to assess a work units climate for quality 
reflected by its quality oriented work processes and practices. 

AIR Paper 2 8 



/ 



Six of the climate indices which emerged from the factor analysis were 
synonymous with the conceptual categories of in the survey instrument. Their 
reliabilities ranged from .89 to .96. Those are the Supportive Unit Climate index 
(.95), Planning for Improvement and Innovation index (.94), Satisfying Those 
Served index (.89), Work Processes index (.94), Unit Efficiency index (.89), and 
Unit Leadership index (.96). 

Three climate indices emerged from the factor analyses of the 
conceptual category related to support for Collecting and Using Information. 
Collecting Information ( reliability of .89) represents a measure of the extent to 
which information is collected within the unit. The Use of Information index 
(reliability of .83) relates to the extent to which information is used within the 
unit. The third index is Role of Information (reliability .94). This index reflects to 
the clarity and utilization of quality standards and the degree to which those 
responsible within a unit are given the information they need to accomplish their 
goals. 

Two climate indices emerged from the factor analyses of the conceptual 
category related to the Unit Staff Members. The first Staff Relations index 
(reliability .93) reflects the dynamics of working relationships within the unit and 
focuses on the activities which are conducive to full participation in the unit. The 
second index, Professional Development (reliability .88), reflects how staff 
members are enabled to develop their full potential as contributors to a quality 
improvement initiative. It focuses on staff members' opportunities to participate 
in education or training on work improvement concepts and techniques. 

Two climate indices emerged from the factor analysis related to the 
conceptual category support for Improving Performance. The first index, 
Supportive Staff Relationships (reliability .96) reflects the working relationships 
among staff members directed toward improving the unit's performance. The 
second index Improving Outputs (reliability .90) includes items that focus on the 
unit's collective efforts to improve results. - 

The final four climate indices emerged from the factor analysis of the 
conceptual category related to support for Unit Performance Indicators. The first 
Overall Performance index (reliability .84) relates to the overall level of unit 
performance when compared to similar units, expectations of those served, unit 
goals, and last year's performance at the same time. The other three indices, 
the Rate of Improvement index (reliability .90), Number of Errors and Mistakes 
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(reliability .89), and Costs of Services indices (reliability .87) all relate to 
comparison of those three dimensions on the same comparisons. 

vu. Discussion and Conclusions 

The research conducted for this paper initially identified thirty five 
potential culture and climate dimensions of work environment emphasizing a 
quality improvement. These dimensions were then incorporated into a 
theoretically based framework containing three quality culture and ten quality 
climate categories for examining the values, practices and procedures that 
support quality improvement in a higher education work environment. This 
conception provided the foundation for then developing a survey instrument to 
measure participants' perceptions of the quality oriented work environment. 

This instrument is the first empirically based measurement instrument 
designed to assess the culture and climate for quality improvement in the non- 
academic work environment in higher education. Twenty seven factor based 
indices emerged which have a high degree of reliability and common content. 
Seventeen indices measure the climate for quality represented by a unit's 
quality oriented work process and support activities. Ten indices measure a 
culture for quality in the work unit. 

The instrument needs to be tested in other settings. The items in the 
questionnaire are higher education oriented and not biased toward any 
particular type of institution. It, however, was designed for the non-academic 
staff and is probably not appropriate for a faculty or student response group. 

The instrument can be used either for building indices of a quality 
oriented work environment or individual items can be examined. Within an 
institution, profiles of the indices can be developed for the entire institution or to 
compare functional groups, job classifications or other work units with data 
appropriately aggregated. It can also be used for making cross institution 
comparisons or assessing change over time in a given institution. 

The instrument and the larger model suggest more conceptually oriented 
research which can look at the influence of external forces on a quality oriented 
environment, the success of different strategies of introducing quality, the 
resulting impact of a quality oriented work environment or the influence of 
individuals on the quality oriented work environment. 
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Clearly, additional studies need to be conducted to increase trie external 
validly and generalizability of this instrument. These studies should include a 
wider array of institutional types. However, there are two primary implications of 
this effort. First, it provides an instrument that provides administrators and 
institutional researchers with a means for better understanding the climate and 
culture for assessing continuous quality improvement in their institution's non- 
academic work environment. Second, it provides higher education 
researchers with a reliable instrument with which to evaluate quality 
improvement initiatives into higher education institutions and to examine the 
impact of a quality oriented work environment. It should help move quality 
improvement in higher education from the realm of belief, promotion and case 
description to an examined, conceptually sound approach to management in 
higher education 
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Table L 

Conceptual Models of Total Quality Management Dimensions 



Author/Model 

W.E. Deming/ 
Fourteen Points 



Juran Institute of 
Quality 



Malcolm Baldridge 
National Quality 
Award Program 

ISO9000 



Crosby Steps 
Cameron Model 



Dimensions 

constancy of purpose toward improvement; adopting new philosophies; 
cease dependence on mass inspection; awarding not based on price 
alone; improve constantly and forever the system of production and 
service; institute training; institute leadership; driving out fear; breaking 
down barriers between staff areas; eliminating slogans; eliminate 
numerical quotas; remove barriers to pride of workmanship: institute a 
vigorous program of education and retraining; take action to accomplish 
the transformation 

thinking about quality planning, establishing quality goals, identifying 
the customers; determine customer needs; provide measurement; 
develop product features, develop process features; transfer process 
controls to operations; application of quality planning 

leadership; information and analysis; strategic quality planning; human 
resource development and management; management of process quality; 
quality and operational results; customer focus and satisfaction 

customer service; process versus results focus; management 
commitment/ responsibility; continuous improvement; less than 20% of 
problems caused by workers; performance measures; croi < functional 
councils create constancy 

management commitment; teamwork; measurement; communication- 
training; goal setting; zero defects 

leadership, information and analysis; quality planning; human resource 
utilization; quality assurance; customer satisfaction; cultural priorities; 
climate; cost containment; quality results; speed and timeliness; 
structure; innovation; quality culture 
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Tabled 

Conceptual Models of Higher Education Quality Improvement Dimensions 



Marchese 



Sherr and Lozier 



Author Dimensions 

focus on quality; customer orientation; continuous improvement; making 
processes work better; extending the mindset; the discipline of 
information; eliminate rework; teamwork; empowering people; training 
and recognition; vision; leadership 

mission and customer focus; systematic approach to operations; 
vigorous development of human resources; long-term thinking; 
institutional commitment 

giving people a voice; less explaining; cutting down steps; a change in 
climate; willingness to "sweat the details"; bringing people together; 
development of a common language; knowing what we are about; 
reduced rework and scrap; "declining dollars" 

commitment to quality, ethic of service, data and analysis management, 
"invisible" data use, respect and listen to all associates, support people 
in their jobs, teamwork, error prevention, 

Qualtec / M-Quality pursuing continuous improvement, respecting people and ideas, 

managing by fact, satisfying those served, teamwork, planning, and 
quality in daily activities, quality environment 

committed presidential leadership; process orientation; involvement of 
all personnel 



Seymour 



Chaffee 



Sylwester and 
Harris 
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APPENDIX 2a 

Factor Analysis of the Work Environment: 
Description of Culture Indices 



Index 


Items 


V-/L1.1EI# /KJ 


R alia n 






Variance 




Unit Philosophy 


5 


57.5% 


.81 


Status Quo 


6 


50.5% 


.80 


Error Detection 


6 


30.6% 


.54 


Error Prevention 


6 


39.7% 


.70 


Continuous 


6 


46.3% 


.76 


Improvement 






Clan Culture 


4 


42.7% 


.53 


Adhocracv Culture 


A 


a c no/ 
43.U /o 


.58 


Hierarchy Culture 


4 


70.5% 


.58 


Market Culture 


4 


53.5% 


.71 
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Index 

Change /Leading 
Culture 

Change /Adapting 
Culture 

Change/ 
Responding 
Culture 

Change / Resisting 
Culture 



Items Cum. % Reliability 

Variance 



1 



1 



1 
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APPENDIX 2b 

Factor Analysis of the Work Environment: 
Description of Climate Indices 

Index Items Cum. % Reliability 

Variance 

Supportive Unit 15 58 4 o /o 95 
Climate 

Planning for n 63 4 o /o 94 

Improvement 
and Innovation 

Satisfying Those 8 572 o/ 0 >89 
Served 

Supportive Work 13 58>1 o /o 94 
Processes 

Collect 9 415 o /o S9 
Information 

Use Information 9 60.4% 83 

Roleof 6 63.4% .94 

Information 



Index Items Cum. % Reliability 



Unit Efficiency 


7 


Variance 

AD A 0 / 
OU.O /o 


on 

.89 


Unit Leadership 


9 


73.3% 


.96 


Staff Relations 


11 


54.7% 


.93 


Professional 
Development 


O 


£1 70/ 
Ol./To 


.88 


Supportive Staff 
Relationships 


1 ^ 


61.6% 


.96 


Improvine 
Outputs 


/ 


67.3% 


.90 


Overall 
Performance 


4 


67.9% 


.84 


Rate of 
Improvement 


4 


76.6% 


.90 


Number of Errors 
and Mistakes 


4 


75.3% 


.89 


Cost of Services 


4 


72.6% 


.87 
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APPENDIX 3 
DESCRIPTIONS OF THE INDICES 



Culture and Climate of a Quality Oriented Work Environment 



Quality Culture (ten indices): 



1. Philosophy: five items related to the way staff members in a particular unit 

fundamentally feel about quality improvement in their work environment; 
quality philosophy includes staff members' perceptions of their mission 
and purpose, willingness to change and improve, propensity to interact 
and share success stories, priorities relating to quality, and concerns for 
quality. 

2. Status Quo: six items related to staff members' perceptions that there have 

been no changes in the unit improvement orientation; status quo includes 
staff members' perception that unit leadership has done nothing to 
enhance quality improvement, quality is not measured, there is no attempt 
to provide quality training, little thought is given to those served, to teams 
exist, and that approach to costs is the same as always 

3. Error Detection: six items related to staff members' perception that there have 

been some changes in the unit improvement orientation; this index 
focuses on the measur ement of errors and mistakes; error detection 
includes staff perceptions that unit leaders identify improvement areas, 
errors and mistakes are measured, team formation is supported, quality 
training is provided upon request, attempts are made to meet the needs of 
those served, across the board cuts are made to meet budget goals. 

4. Error Prevention: six items related to staff members' perception that there 

have been ongoing changes in the unit improvement orientation; this 
index focuses on the prevention of errors and mistakes; includes staff 
perceptions that leaders try to prevent mistakes, errors, and budget 
inefficiencies, team formation is actively encouraged, and attempts are 
made to exceed the expectations of those we serve. 

5. Continuous Improvement: six items related to staff members' perception that 

there have been constant efforts to address unit improvement orientation; 
this index focuses on the units' continuous striving for improvement; 
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continuous improvement includes staff perceptions that leaders are 
continuously raising performance standards, expectations of those served 
are exceeded, cost are reduced without any effect to service provided, and 
almost all staff members work in teams. 



6. Clan: four items related to staff members' perception that their unit has a 

strong sense of cohesion and teamwork; clan culture orientation includes a 
team oriented management style, loyalty and commitment as unit wide 
characteristics, unit success defined by professional development, 
teamwork, and concern for people, and a leadership style that is mentor 
oriented. 



7. Adhocracy: four items re. 
culture orientation in 
innovative leadership 
and innovative efforts, 
individual initiative. 



a loosely coupled unit culture; adhocracy 
focus on innovation and creativity, a 
\it success defined by cutting edge outputs 
anagement style characterized by 



8. Hierarchy: four items rented eaucratic hierarchical unit culture; 

hierarchy culture orientatk iudes a security oriented unit 
management style, a focus on policies and procedure that promote 
stability, success defined by efficiency and stability, and an organizing 
leadership style. 

9. Market: four items related to a results oriented unit culture; market culture 

orientation includes an achievement oriented management style, a hard- 
driving leadership style, success defined by competitiveness, and goal 
accomplishment as the unifying cultural factors. 

10. Change: four items related to a unit culture of change; change culture 

orientation includes leading , adapting to, responding, and resisting 
change within the unit. 



Quality Climate (seventeen indices) 



11. Supportive Unit Climate: fifteen items that relate to the general atmosphere 
or mood within the work unit; unit climate focuses on how employees feel 
about their daily interactions with co-workers, leaders, and the work itself; 
it also refers to the sense of cooperation, teamwork, trust enjoyment and 
the feeling of being valued, accepted, and sought after for input into 
decision-making. 
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12. Planning for Improvement and Innovation: eleven items related the unit 

climate of planning improvement and innovation; this index refers to unit 
improvement plans and the extent to which they are known, understood, 
and utilized by individual unit members; it also includes the units focus 
on the degree to which employees seek innovation and are rewarded for 
successful developments; the units ability to continuously improve its 
planning processes is also included. 

13. Satisfying Those Served: eight items related to service to the units' clients, 

beneficiaries, stakeholders, constituents, service recipients, or customers; 
satisfying those that are served by the unit encompasses the identification 
of internal and external customers, the understanding of customers needs 
and expectations, procedures in place to gather customer input and utilize 
the information for improvement, and a general focus on anticipating and 
satisfying those who are served. 

14. Supportive Work Processes: thirteen items related to the key elements of 

improving the quality and design of basic day-to-day work processes; 
this index includes understanding and improving processes which are 
both problematic and problem-free; it focuses on process assessment, 
cycle time reduction, efficiency, and effectiveness; it also examines the 
scope and effectiveness of process improvement. 

15. Collect Information: nine items related to the extent to which information is 

collected within the unit; this index assesses the degree to which 
information is collected on the following: individual and unit successes, 
achievements, innovative ideas, timeliness, mistakes, complaints, 
employee performance, customer and employee satisfaction, 
benchmarking, and the data collection process. 

16. Use Information: nine items related to the extent to which information is 

used within the unit; this index assesses the degree to which information 
is used on the following: individual and unit successes, achievements, 
innovative ideas, timeliness, mistakes, complaints, employee 
performance, customer and employee satisfaction, benchmarking, and the 
data collection process. 

17. Role of Information: six items related to the clarity and utilization of quality 

standards and the degree to which those most responsible are given the 
required information. 

18. Unit Efficiency: seven items related to the ways in which work is 

organized to increase the unit's efficiency; this index focuses on the 
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reduction of time and resources used to accomplish a unit's work; it 
includes reducing unproductive employee time, reducing cycle time to 
respond to those served, reducing costs and waste, simplifying work 
processes and developing time standards for work activities. 

19. Unit Leadership: nine items related to unit leaders' personal leadership and 

involvement in developing a unit which continuously improves quality 
and has a clear understanding of quality values; this index focuses on 
leadership communication skills, managing by fact, being open to new 
ideas and change, removing barriers and obstacles, reallocating resources 
based on quality values, and leading by example. 

20. Staff Relations: eleven items related to the dynamics of working 

relationships within the unit; this index focuses on the key elements 
which are conducive to full participation in unit activities, it i icludes 
staff members being treated with dignity and respect, seeking diversitv 
of opinion, staff participation in decision-making and decision 
implementation, understanding expectations, understanding quality 
philosophy within the unit, and maintaining optimum unit resources. 

21. Professional Development: six items related to the key elements of how the 

human resources are enabled to develop their full potential as contributors 
to continuous quality improvement; this index focuses on staff members' 
opportunities to participate in education or training on work 
improvement concepts and techniques, rewards for achieving quality 
objectives, opportunities to participate in personal and professional 
growth and development, and improvement of human resource 
development programs. 

22. Supportive Staff Relationships: thirteen items related to the working 

relationships of staff members with specific regard to improving the unit's 
performance; this index includes leaders increasing trust among staff 
members, staff loyalty increasing, helping co-workers to improve, 
decreasing complaints of unit leadership, improving unit communication 
decreasing conflict within the unit, and increasing cooperation with other' 
units. 

23. Improving Outputs: seven items related to improving the results of the 

unit's collective efforts; this index includes improvement of the quality of 
work, improvement of outputs and services, decrease in customer 
complaints, continually setting higher work standards, decrease in time 
required to introduce new services, and decrease in missed deadlines. 
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24. Overall Performance: four items related to the overall level of performance 

when compared to similar unit, expectations of those served, unit goals, 
and last year's performance at the same time. 

25. Rate of Improvement: four items related to the rate of improvement when 

compared to similar unit, expectations of those served, unit goals, and last 
year's performance at the same time. 

26. Number of Errors and Mistakes: four items related to the number of errors 

and mistakes when compared to similar unit, expectations of those served, 
unit goals, and last year's performance at the same time. 

27. Cost of Services: four items related to the cost of services when compared to 

similar unit, expectations of those served, unit goals, and last year's 
perfc-mance at the same time. 
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APPENDIX 4a 

Comparison Of Conceptual Categories And Factor Indices 
Of A Quality Culture And Climate 



CULTURE 



Literature Based 
Conceptual Categories 
In Framework 



Indices 

Emerging From 
Factor Analysis 



Quality Philosophy 



Unit Philosophy 



Quality Improvement 
Culture 



Status Quo 

Error Detection 

Error Prevention 

Continuous 
Improvement 



Organizational Culture Clan Culture 

Adhocracy Culture 
Hierarchy Culture 
Market Culture 
Change Culture 



o 
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APPENDIX 4b 



Comparison Of Conceptual Categories And Factor Indices 
Of A Quality Culture And Climate 

CLIMATE 



Literature Based 
Conceptual Categories 
In Framework 



Indices 

Emerging From 
Factor Analysis 



Unit Climate 



Supportive Unit 
Climate 



Planning For 
Improvement 



Planning for 
Improvement 
and Innovation 



Satisfying Those 
Served 



Satisfying Those 
Served 



Work Processes 



Supportive Work 
Processes 



Collecting and Using 
Information 



Collect Information 



Use Information 
Role of Information 
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CLIMATE 



Literature Based 
Conceptual Categories 
In Framework 



Indices 

Emerging From 
Factor Analysis 



Unit Efficiency 



Unit Efficiency 



Leadership 



Unit Leadership 



Unit Staff Members 



Sta/f Relations 

Professional 
Development 



Improving Performance 



Supportive Staff 
Relationships 

Improving Outputs 



Unit Performance 
Indicators 



Rate of 
Improvement 

Overall 
Performance 

Number of 
Errors/Mistakes 

Costs of Services 



